fix: prevent false spawn failures for live agents - #47
Open
ruby-dlee wants to merge 8 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes six observed false spawn failures on 2026-07-29 across the Firstmate and Relvino pools, including this task's own Codex spawn. In every incident the worktree-path readiness probe timed out even though the harness agent was provably alive and working.
The root cause was shared with
fm-herdr-send-fix-h7: Herdr path reads were gated by a legacy reachability/certificate path that could reject a healthy native server. This branch builds on that fix rather than duplicating it.What changed
fm-spawn.shnow independently asks the backend's conservative harness-agent liveness classifier before condemning an unreadable endpoint.alivecommits the spawn through the normal success path, with nodirect_spawn_cleanup,direct_spawn_artifacts, orrollback_pendingpoison markers.deadorunknownagent still enters the existing rollback path.mainbranch in CI-created clones.Verification
Implementation-specific automated coverage is green:
FM_TEST_FOCUSED=spawn-readiness bash tests/fm-account-directory.test.shpasses the live-agent success, genuinely dead rollback, and wrong-worktree rollback cases.tests/fm-account-directory.test.shaffected suite passes.FM_TEST_FOCUSED=readsteer-native-cutover bash tests/fm-backend-herdr.test.shpasses the coordinated Herdr read/steer regressions.Live validation limitation
No live end-to-end Herdr spawn was run because this task was not scaffolded with
--herdr-lab, and its safety contract forbids touching Herdr lifecycle without that guard.That leaves real backend timing, native process registration, and the post-land behavior of an ordinary production spawn unproven by this branch. The real-world reproduction is unusually strong—six independent incidents across two pools, all with active Codex agents—but it is reproduction evidence, not post-fix live validation.
Firstmate will verify the next ordinary spawn after landing and report whether it records clean success metadata. An isolated Herdr-lab lifecycle regression is being filed separately.
Risk
The fallback is deliberately narrow: only confident harness-agent liveness can rescue an unreadable readiness probe. Concrete isolation violations and dead or unknown endpoints still fail closed and roll back.